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(54) Sending progressive video sequences 

(57) In encoding and decoding video signals, a pro- 
gressive video bitstream (201 ) is received which has ref- 
erence frames and non-reference frames, each having 
an initial temporal reference in accordance with an initial 
frame sequence structure. The temporal references of 



only reference frames are remapped (202), by ignoring 
the non-reference frames. The reference frames are 
packetized (21 0-21 3) with a base packet-identifier (PID) 
and the non-reference frames with an enhancement 
PID, to provide base and enhancement transport bit- 
streams, respectively. 
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Description 



[0001] The present invention relates to video process- 
ing systems, and, in particular, to apparatuses and 
methods for encoding video sequences in a bitstream 
which is backward compatible for decoding into lower 
quality video by older decoders and which may be de- 
coded into high-quality progressive video by newer de- 
coders compatible with the high-quality encoding 
[0002] Data signals are often subjected to computer 
processing techniques such as data compression or en- 
coding, and data decompression or decoding. The data 
signals may be, for example, video signals. Video sig- 
nals are typically representative of video pictures (imag- 
es) of a motion video sequence. In video signal process- 
ing, video signals are digitally compressed by encoding 
the video signal in accordance with a specified coding 
standard to form a digital, encoded bitstream. An encod- 
ed video signal bitstream may be decoded to provide 
decoded video signals. 

[0003] The term "frame" is commonly used forthe unit 
of a video sequence. A frame contains lines of spatial 
information of a video signal. Depending on the encod- 
ing format, a frame may consist of one or more fields of 
video data. Thus, various segments of an encoded bit- 
stream represent a given frame or field. The encoded 
bitstream may be stored for later retrieval by a video de- 
coder, and/or transmitted to a remote video signal de- 
coding system, over transmission channels or systems 
such as Integrated Services Digital Network {ISDN) and 
Public Switched Telephone Network (PSTN) telephone 
connections, cable, and direct satellite systems (DSS) 
[0004] Video signals are often encoded, transmitted 
and decoded for use in television (TV) type systems' 
Many common TV systems, e.g. in North America op- 
erate in accordance with the NTSC (National Television 
Systems Committee) standard, which operates at 
(30*1000/1001) - 29.97 frames/second (fps). The spa- 
tial resolution of SDTV is sometimes referred to as 
SDTV (standard definition TV). NTSC originally used 30 
fps to be half the frequency of the 60 cycle AC power 
supply system. It was later changed to 29.97 fps to throw 
it "out of phase" with power, to reduce harmonic distor- 
tions. Other systems, such as PAL (Phase Alternation 
by Line), are also used, e.g. in Europe. 
[0005] In the NTSC system, each frame of data is typ- 
ically composed of an even field interlaced or inter- 
leaved with an odd field. Each field consists of the pixels 
in alternating horizontal lines of the picture or frame Ac- 
cordingly, NTSC cameras output 29.97x2=59 94 fields 
of analog video signals per second, which includes 
29.97 even fields interlaced with 29.97 odd fields to pro- 
vide video at 29.97 fps. NTSC images typically have a 
resolution of approximately 720 (h) x 480 (v) active pix- 
els. Thus, each field is 720 x 240, to provide interlaced 
frames of 720 x 480. These specifications are provided 
in CCIR Rec. 601 , which specifies the image format ac- 
quisition semantic, and parts of the coding for digital 
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"standard" television signals. ("Standard" television is in 
the resolution of PAL, NTSC, and SECAM.) 
[0006] Various video compression standards are 
used for digital video processing, which specify the cod- 
ed bitstream for a given video coding standard These 
standards include the International Standards Organi- 
zation/International Electrotechnical Commission (ISO/ 
IEC) 11172 Moving Pictures Experts Group-1 interna- 
tional standard ("Coding of Moving Pictures and Asso- 
ciated Audio for Digital Storage Media") (MPEG-1) and 
the ISO/IEC 13818 international standard ("Generalized 
Coding of Moving Pictures and Associated Audio Infor- 
mation") (MPEG-2). Another video coding standard is 
H.261 (Px64), developed by the International Telegraph 
Union (ITU). In MPEG, the term "picture" refers to a bit- 
stream of data which can represent either a frame of 
data (i.e., both fields), or a single field of data. Thus 
MPEG encoding techniques are used to encode MPEG 
"pictures" from fields or frames of video data MPEG-1 
was built around the Standard Image Format (SIF) of 
352x240 at 30 frames per second (fps). MPEG data 
rates are variable, although MPEG-1 was designed to 
provide VHS video quality at a data rate of 1 .2 megabits 
per second, or 150 KB/sec. In the MPEG-1 standard 
video is strictly non-interlaced (i.e. progressive) For 
progressive video, the lines of a frame contain samples 
starting from one time instant and continuing throuqh 
successive lines to the bottom of the frame 
[0007] MPEG-2, adopted in the Spring of 1994 is a 
compatible extension to MPEG-1, which builds on 
MPEG-1 and also supports interlaced video formats and 
a numberof otheradvanced features, including features 
to support HDTV (high-definition TV). MPEG-2 was de- 
signed, in part, to be used with NTSC-type broadcast 
TV sample rates using the CCIR Rec. 601 (720 sam- 
ples/line by 480 lines per frame by 29.97 fps In the in- 
terlacing employed by MPEG-2, a frame is split into two 
fields, a top field and a bottom field. One of these fields 
commences one field period later than the other Each 
video field is a subset of the pixels of a picture transmit- 
ted separately. MPEG-2 is a video encoding standard 
which can be used, for example, in broadcasting video 
encoded in accordance with this standard. The MPEG 
standards can support a variety of frame rates and for- 
mats. 

[0008] Motion compensation is commonly utilized in 
video signal processing. Motion compensation tech- 
niques exploit the temporal correlation that often exists 
between consecutive pictures, in which there is a ten- 
dency of some objects or image features to move within 
restricted boundaries from one location to another from 
picture to picture. In the MPEG standards, such as the 
MPEG-2 standard, there may be different picture or 
frame types in the compressed digital stream, such as 
I frames, P frames, and B frames. I frames, or intra- 
frames, are self-contained, that is, they are not based 
on information from previously transmitted and decoded 
frames. Video frames which are encoded with motion 
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compensation techniques are referred to as predicted 
frames, or P frames, since theircontent is predicted from 
the content of previous I or P frames. P frames may also 
be utilized as a base for a subsequent P frame. I and P 
frames are both 'anchor" frames, since they may be 
used as a basis fcr otherf rames, such as B or P frames 
which are predicted based on anchor frames. A "bidi- 
rectional" or B frg.me is predicted from the two anchor 
frames transmitted most recently relative to the trans- 
mission of the B frame. Other standards, such as H.261 , 
utilize only I and P frames. 

[0009] Most MPEG encoding schemes use a twelve- 
to fifteen-compressed frame sequence called a group 
of pictures (GOP ). Each GOP typically begins with an I 
frame, and optionally includes a number of B and P 
frames. The parameter M is often used to represent the 
distance between P frames in a GOP, and the parameter 
N represents the total number of frames in a GOP (i.e., 
the distance between I frames in consecutive GOPs). 
[0010] An MPEG bitstream typically contains one or 
more video streams multiplexed with one or more audio 
streams and other data, such as timing information. In 
MPEG-2, encoded data which describes a particular 
video sequence is represented in several nested layers: 
the Sequence layer, the GOP layer, the Picture layer, 
the Slice layer, and the Macroblock layer. To aid in trans- 
mitting this infomnation, a digital data stream represent- 
ing multiple video sequences is divided into several 
smaller units and each of these units is encapsulated 
into a respective packetized elementary stream (PES) 
packet. For transmission, each PES packet is divided, 
in turn, among a plurality of fixed-length transport pack- 
ets. Each transport packet contains data relating to only 
one PES packet. The transport packet also includes a 
header which holds control information to be used in de- 
coding the transport packet. 

[0011] Thus, :he basic unit of an MPEG stream is the 
packet, which includes a packet header and packet da- 
ta. Each packet may represent, for example, a field of 
data. The packet header includes a stream identification 
code and may include one or more time-stamps. For ex- 
ample, each data packet may be over 100 bytes long, 
with the first two 8-bit bytes containing a packet-identi- 
fier (PID) field. In a DSS application, for example, the 
PID may be a SCID (service channel ID) and various 
flags. The SCID is typically a unique 12-bit number that 
uniquely identities the particular data stream to which a 
data packet belongs. Thus, each compressed video 
packet contains a PID such as a SCID. 
[0012] Wher an MPEG-2 encoded image is received 
by a video decoding system, a transport decoder de- 
codes the transport packets to reassemble the PES 
packets. The PES packets, in turn, are decoded to re- 
assemble the MPEG-2 bitstream which represents the 
image. A given transport data stream may simultane- 
ously convey multiple image sequences, for example as 
interleaved transport packets. 

[0013] For example, an MPEG-2 encoded video bit- 



stream may be transported by means of DSS packets 
when DSS transmissions are employed. Most DSS vid- 
eo programs are encoded at 544 pixels/line and 480 
lines/frame. All 29.97 frames/sec are coded. The exact 
5 number of coded frames/sec depends on the exact se- 
quence. DSS systems allow users to receive directly 
many TV channels broadcast from satellites, with a DSS 
receiver. The receiver typically includes a small 1 8-inch 
satellite dish connected by a cable to an integrated re- 
w ceiver/decoder unit (IRD). The satellite dish is aimed to- 
ward the satellites, and the IRD is connected to the us- 
er's television in a similar fashion to a conventional ca- 
ble-TV decoder. In the IRD, front-end circuitry receives 
a signal from the satellite and converts it to the original 
15 digital data stream, which is fed to video/audio decoder 
circuits which perform transport extraction and decom- 
pression. For MPEG-2 video, the IRD comprises an 
MPEG-2 decoder used to decompress the received 
compressed video. 
20 [0014] In MPEG-2, four different "profiles" are de- 
fined, each corresponding to a different level of com- 
plexity of the encoded image, e.g. the image/picture res- 
olution. Each profile define the colorspace resolution 
and scalability of the bitstream. For each profile, differ- 
25 ent levels are defined, each level corresponding to a dif- 
ferent image resolution. The various levels for a given 
profile define the maximum and minimum for image res- 
olution, and Y (luminance) samples per second, the 
number of video and audio layers supported for scalable 
30 profiles, and the maximum bit rate per profile. The com- 
bination of a profile and a level produces an architecture 
which defines the ability of a decoder to handle a par- 
ticular bitstream. 

[0015] The most common profile for broadcast appli- 
35 cations is the main profile (MP) format. One of the 
MPEG-2 "standards," known as Main Profile, Main (or 
Medium) Level (MP@ML) is intended for encoding vid- 
eo signals conforming to existing SD television stand- 
ards (i.e., NTSC and PAL). This standard may be used 
40 to encode video images having 480 active lines each 
with 720 active pixels with a 2:1 interlace scan. When 
the horizontal and vertical blanking intervals are added 
to these signals, the result has 525 lines by 858 pixels. 
When they are decoded, and displayed with a 13.5 MHz 
45 display clock signal, these signals produce images that 
correspond to NTSC-type broadcast images. Another 
standard, known as Main Profile, High Level (MP@HL), 
is intended for encoding HDTV images. 
[0016] As the quality of some systems such as TV 
so systems improves, it is desirable to provide HD-compat- 
ible encoded video signals for video transmissions. 
However, there may be both SD and HD receivers, and 
the SD receivers and systems may not be compatible 
with the improved transmission/encoding standard, i.e. 
55 the improved standards may not be "backward-compat- 
ible". For example, conventional SD DSS IRDs are not 
able to decode any formats better than MP@ML for- 
mats. Thus, some DSS systems are forced to transmit 
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an HD channel of data, as well as an SD version of the 
HD channel, so that the DSS SD receivers can receive 
and decode the transmission. This is a very expensive 
solution since it takes a complete SD channel bandwidth 
in addition to the HD channel bandwidth. Bandwidth is 
wasted since redundant information is transmitted 
There is a need, therefore, for techniques for encoding 
and transmitting improved or enhanced signals which 
are also backward compatible with the prior standard 
to avoid having to transmit redundant or extra channels 
of data. 

[0017] In the present invention, a method and appa- 
ratus are provided for encoding and decoding video sig- 
nals. A progressive video bitstream is received which 
has reference frames and non-reference frames, each 
having an initial temporal reference in accordance with 
an initial frame sequence structure. The temporal refer- 
ences of the only the reference frames are remapped 
by ignoring the non-reference frames. The reference 
frames are packetized with a base packet-identifier 
(PID) and the non-reference frames with an enhance- 
ment PID, to provide base and enhancement transport 
bitstreams, respectively. 



BRIEF DE SCRIPTION OF THE DRAWINGS 
[0018] 

Fig. 1 shows a digital video system for encoding, 
transmitting, and decoding video image signals, ac- 
cording to a preferred embodiment of the present 
invention; 

Fig. 2 is a flow diagram illustrating the encoding per- 
formed by the video encoder of the video system of 
Fig. 1; 

Fig. 3 illustrates the bitstream packetizing per- 
formed by a DSS transport packetizer of the video 
encoder of Fig. 1 ; 

Fig. 4 illustrates the base stream-only decoding op- 
eration performed by a standard DSS video decod- 
er of a system such as the system of Fig. 2; and 
Fig. 5 illustrates the progressive video decoding op- 
eration performed by a modified DSS video decoder 
of a system such as the system of Fig. 2. 

[0019] In the present invention, an MPEG-2 type en- 
coding process is modified to provide an encoded bit- 
stream that is backward-compatible with existing 
MPEG-2 DSS SD IRDs and which provides additional 
information to suitably modified decoders to provide 
higher quality video. 

[0020] Referring now to Fig. 1 , there is shown a digital 
video system 100 for encoding, transmitting, and decod- 
ing video image signals, according to a preferred em- 
bodiment of the present invention. Video encoder 110 
receives an video stream from a video data source such 
as a video camera. This may be a high-quality, 59 94 Hz 
(fps) input video data stream. As described in further de- 



tail below with reference to Fig. 2, video encoder en- 
codes this data with an encoder such as an MPEG-2 
encoder to generate a progressive video bitstream This 
may be, for example, a 59.94 Hz (fps) progressive bit- 
5 stream. This bitstream is then split by further functional 
components of encoder 110, such as the transport en- 
coder (transport IC), into two streams, a base stream 
and an enhancement stream, and transmitted over a 
communications medium such as DSS channel 120 as 
™ a transport bitstream. 

[0021] The transport IC makes the base stream back- 
ward compatible with existing decoders, such as DSS 
SD IRDs, and compliant with MPEG-2 MP@ML syntax 
in an embodiment. Thus, a standard, non-modified DSS 
SD IRD is able to receive the base stream portion of the 
compressed progressive video. The enhanced video 
transmission is thus transparent to non-upgraded SD re- 
ceivers and decoders. That is, the enhanced video 
transmission is backward-compatible with the previous 
20 existing SD format. A suitably modified MP@HL type 
video decoder and transport decoder can be used to de- 
code the base and enhancement streams to provide 
high-quality progressive video. An HDTV or EDTV (en- 
hanced definition television) decoding system may be 
25 used for this purpose, for example. 

[0022] In an embodiment, the base stream has a 
frame rate of 29.97 Hz (fps), i.e. the SDTV rate. The 
enhancement stream, combined with the base stream 
can be decoded by an EDTV decoder, for example to 
30 provide a 59.94 Hz progressive bitstream for EDTV sys- 
tems. Thus, the encoder 110 provides a combined 
stream having base and an enhancement streams To- 
gethertheseform an enhanced transmission, which can 
be received and decoded by an enhanced receiver and 
35 decoder, such as HDTV or EDTV, to provide progressive 
video (e.g. 59.94 Hz). The base stream may be used by 
itself by an MP@ML type system, such as SDTV, to re- 
trieve only the 29.97 Hz (fps) MP@ML video informa- 
tion. 

40 [0023] Thus, a video decoder 1 30 receives the trans- 
port bitstream, and provides demodulation and decod- 
ing to provide some useful output, e.g. a display on a 
monitor(notshown). In one embodiment, video decoder 
130 is a conventional DSS IRD, which extracts the base 
4 $ stream and decodes it to provide SDTV 29.97 fps inter- 
laced video. In another embodiment, decoder 130 is an 
HDTV or EDTV decoder in accordance with the present 
invention, which extracts and decodes both the base 
and enhancement stream to provide 59.94 fps proqres- 
50 sive video for EDTV applications. 

[0024] In the following discussion, a DSS embodi- 
ment employing SCIDs is employed and reference is 
therefore made to SCIDs, although it will be understood 
that in general a PID may be utilized. In an embodiment 
55 the elementary base stream is an M P EG-2 M P @ ML vid- 
eo bitstream, having a GOP bitstream structure with the 
following encoded ordering: 
10 P1 P2 ... P14 
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[0025] In this GOP, M=1, N=15, and the GOP bit- 
stream length in time is 15 frames/29.97 fps = 0.5 sec. 
The time distance between two displayed frames = 
1/29.97 sec, which is the same interval as for NTSC vid- 
eo. In an embodiment, this elementary stream is pack- 
etized into a DSS transport stream with one SCID (SCID 
»)• 

[0026] The enhancement stream is not an MPEG-2 
compliant video bitstream. In an embodiment, it consists 
of B frames only and has the following structure: 
B0 B1 B2 ... B14 

[0027] For the enhancement bitstream, as for the 
base bitstream, N=15. The time distance between two 
displayed frames is 1/29.97 sec. The enhancement bit- 
stream is packet zed into a DSS transport stream with 
a different SCID than that of the base bitstream (i.e., 
SCIDj). 

[0028] Referring now to Fig. 2, there is shown a flow 
diagram 200 illustrating the encoding performed by vid- 
eo encoder 110 Df video system 100, which generates 
the video transport bitstream transmitted via DSS chan- 
nel 1 20. I n an embodiment, video encoder 1 1 0 compris- 
es an MPEG-2 encoder, which provides a 59.94 Hz pro- 
gressive video bitstream (step 201). This progressive 
video bitstream has a GOP structure of M=2, N=30, with 
the following general GOP structure ordering: 

B0 11 B2P3B4P5B6 P7 B8 P9B10P11 B12P13 
B14 P15 B16 PI 7 B18 

P19 B20 F21 B22 P23 B24 P25 B26 P27 B28 P29 
[0029] Encoder 1 1 0 remaps the temporal reference of 
the reference (I and P) frames in the base stream as 
follows (step 202): 
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Pn => P(n-1)/2 



[0030] The purpose of the remapping of the reference 
frames is so that the base stream , which will consist only 
of these reference frames, will have consecutive tem- 
poral rerefences and be self-contained from the point of 
view of an MP<^ML decoder, i.e. backward compatible 
with the MP@ML decoder despite the presence of the 
enhancement stream. 

[0031] The transport IC of video encoder 110 then 
separates the I , P frames from the B frames (step 210), 
for the base and enhancement bitstreams, respectively, 
and assigns either SCID i orSCIDj, respectively, to each 
packet of a particular frame (steps 211, 212). A DSS 
transport packetizer of video encoder 1 1 0 then packetiz- 
es the bitstreams as illustrated in Fig. 3 (step 213). As 
shown in Fig. 3, for example, the various packets of 
frame 10 (only two of which are illustrated in Fig. 3) are 
assigned SCID i. 

[0032] Next some of the transport stream parameters 
in the base bitstream are remapped (step 220) so as to 



make the base bitstream compliant with DSS transport 
specifications. 

[0033] The base and enhancement bitstreams, con- 
taining their respective SCIDs i, j, are then transported 
5 via DSS channel 120, where the packets are received 
by video decoder 130 which contains or is coupled to a 
satellite dish for receiving the DSS signals over the DSS 
channel 120, and which comprises an IRD. The IRD it- 
self contains video/audio decoder circuits which perform 
w transportextractionanddecompression,i.e.,atransport 
decoder and an MPEG video decoder. 
[0034] If video decoder 130 is a standard MPEG-2 
DSS compatible (MP@ML) decoder, it cannot decode 
the entire two-part bitstream to provide progressive vid- 
15 eo. However, because the enhanced bitstream is back- 
ward-compatible with existing MPEG-2 DSS type 
MP@ML decoders, decoder 130 is able to decode the 
base stream. Because the base stream is an MP@ML 
stream, DSS video decoder 130 is able to decode it to 
20 display it as a 59.94 Hz interlaced video sequence. This 
base stream decoding operation is illustrated in Fig. 4, 
where packets denoted SCID i are base stream packets 
and packets denoted SCID j are enhancement stream 
packets. As shown, the I and P frame-packets, which 
25 contain the SCID i, are extracted by the transport de- 
coder circuitry (IC) of decoder 130, and then decoded 
by an MP@ML-compatible MPEG-2 decoder unit of de- 
coder 130. For example, the two SCID i packets illus- 
trated for frame 1 0, which are part of the base stream, 
30 as well as the two SCID i packets illustrated for frame 
P1 , are extracted by the transport decoder IC and then 
decoded by an MP@ ML video decoderto provide SDTV 
quality video. 

[0035] Alternatively, video decoder 130 may be a 
35 modified MP@HL-1440 DSS decoder having transport 
extraction and decoding units able to take full advantage 
of the entire two-part bitstream to provide progressive 
video. Thus, if video decoder 1 30 can decode 59.94 Hz 
progressive video sequences, it extracts both the base 
40 and enhancement streams and decodes them to pro- 
vide 59.94 Hz progressive video, as illustrated in Fig. 5. 
[0036] Thus, in the present invention, the encoding of 
the present invention ensures that the base stream 
looks just like individual SD streams so that SD IRDs 
45 can decode them, while specially-modified HD I RDs can 
decode both the base and enhancement streams to pro- 
vide progressive video. Accordingly, the present inven- 
tion provides a method to use a non-scalable MPEG 
syntax is to support a "scalable" transmission, without 
50 requiring special syntax to support the "scalability" (the 
special syntax would otherwise prevent non-modified 
SD IRDs from being able to decode data from the trans- 
mitted bitstream). 



55 



Claims 

1. A method for encoding video signals, character- 
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ized by the steps of: 



(a) receiving a progressive video bitstream 
comprising reference frames and non-refer- 
ence frames, each having an initial temporal 
reference in accordance with an initial frame 
sequence structure; 

(b) remapping the temporal references of the 
reference frames by ignoring the non-reference 
frames; and 

(c) packetizing the reference frames with a 
base packet-identifier (PID) and the non-refer- 
ence frames with an enhancement PID, to pro- 
vide base and enhancement transport bit- 
streams, respectively. 
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2. The method of claim 1 , further characterized by the 

step of extracting and decoding, with an MP@ML 
decoder, only packets having the base PID, to pro- 
vide an MP@ML decoded video bistream. 20 

3. The method of claim 1 , further characterized by the 

step of extracting and decoding, with an MP@ML 
decoder, packets having both the base PID and the 
enhancement PID, to provide the base and en- 25 
hancement bitstreams, and combining said bit- 
streams to provide an MP@HL decoded video bis- 
tream. 



The apparatus of claim 7, further characterized by 
an MP@ML decoder for extracting and decoding 
only packets having the base PID, to provide an 
MP@ML decoded video bistream. 

The apparatus of claim 7, further characterized by 

an MP@HL decoder for extracting and decoding 
with an MP@ML decoder, packets having both the 
base PID and the enhancement PID, to provide the 
base and enhancement bitstreams, andfor combin- 
ing said bitstreams to provide an MP@HL decoded 
video bistream. 

10. The apparatus of claim 7, characterized in that 

said reference frames comprise I and P frames and 
said non-reference frames comprise B frames. 

1 1 . The apparatus of claim 7, characterized in that the 

remapper remaps the temporal references of the 
reference frames so that the reference frames are 
consecutively numbered. 

12. The apparatus of claim 7, characterized in that 

each PID is a service channel identifier (SCID). 



The method of claim 1 , characterized in that said 30 
reference frames comprise I and P frames and said 
non-reference frames comprise B frames. 

The method of claim 1 , characterized in that step 
(b) comprises the step of remapping the temporal 35 
references of the reference frames so that the ref- 
erence frames are consecutively numbered. 

The method of claim 1 , characterized in that each 
PID is a service channel identifier (SCID). 40 

An apparatus (110) for encoding video signals 
characterized by: 

(a) a remapper (202) for receiving a progres- 45 
sive video bitstream comprising reference 
frames and non-reference frames, each having 

an initial temporal reference in accordance with 
an initial frame sequence structure, and for 
remapping the temporal references of the only 50 
the reference frames by ignoring the non-refer- 
ence frames; and 

(b) a transport packetizer (210-213) for pack- 
etizing the reference frames with a base pack- 
et-identifier (PID) and the non-reference 55 
frames with an enhancement PID, to provide 
base and enhancement transport bitstreams, 
respectively. 
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